Queues and Messages
Foopipes is built around message queues and topic based messaging. It has a built in memory based message queue, or can be configured to use an external durable message queue such as RabbitMQ.
pipelines:
-
when:
- {service: webhook }
then:
- {task: publish, service: queue, topic: build }
-
when:
- {service: queue, topic: build }
do:
- {task: command, command: "bash", args: "build.sh" }
then:
- {task: publish, service: queue, topic: push }
Built in Queue service
Foopipes has a built in memory based message queue service with name queue
. You don't need to configure this service, but it can
be replaced with a different queue of choice.
plugins:
- MyQueuePlugin
services:
queue:
type: myqueueplugin
url: ampq://myhost/
System messages topics
started
an event message fired when a Foopipe instance has been initialized and started.
pipelines:
-
when:
- { service: queue, topic: started }
from:
- { task: httpget, url: "https://registry.npmjs.org/foopipes"}
do:
- { task: select, path: "$.versions.*" }
to:
- { task: file, filename: "foopipe_#{version}.json" }
Publish/await task results
One tricky feature of Foopipes is that a pipeline can forked into a different pipeline right after a do task. It can also put the pipeline on hold until the fork returns.
Under the hood this is accomplished using messages and replies.
pipelines:
-
when:
- { service: queue, topic: started }
from:
- { task: httpget, url: "https://registry.npmjs.org/foopipes"}
do:
- { task: select, path: "$.versions.maintainers[*]", publish: email_maintainer, await: false, keepResult: false }
- { task: select, path: "$.versions.*" }
to:
- { task: file, filename: "foopipe_#{version}.json" }
-
when:
- { service: queue, topic: email_maintainer }
do:
- { task: node, module: emailer, function: sendMail, name: "#{name}", email: "#{email}" }